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DEVICES, METHODS AND SOFTWARE 
FOR GENERATING INDEXING METATAGS IN REAL TIME 
FOR A STREAM OF DIGITALLY STORED VOICE DATA 



BACKGROUND OF THE INVENTION 

1 . Field of the invention. 

The present invention is related to the field of network telephony, and more 
specifically to devices, softwares and methods for indexing telephone conversations 
while they are being recorded. 

2. Description of the related art. 

Networks, such as Local Area Networks (LANs), Wide Area Networks 
(WANs), or the internet are increasingly being used for telephony. Network 
telephony is different from conventional telephony in many aspects. In network 
telephony the voice is encoded digitally, and transmitted as packets. The packets find 
their way through the network by complying with the so-called Internet Protocol (IP). 

A problem in telephony is in recording conversations. These would require a 
massive data storage, since encoding voice consumes large amounts of bandwidth. 

The problem is that needles of precious voice data are often hidden in 
haystacks of whole conversations. For example, in a telephone call with a 
stockbroker, a client's order forms a legally binding request. In the event of a later 
dispute, the client's voice placing the order must be retrieved. But that short duration 
of the telephone call may be buried in a long telephone conversation. For retrieving 
that short duration, it is necessary to play back the entire telephone call, which is time 
consuming. For another example, in a conference call, it may be desirable to isolate 
the instances where only a certain party speaks. 

BRIEF SUMMARY OF THE INVENTION 

The present invention overcomes these problems and limitations of the prior 

art. 

Generally, the present invention provides devices, softwares and methods for 
generating, in real time, indexing metadata for select portions of a telephone 
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conversation or conference. The indexing metadata is generated responsive to inputs 
received while the conversation is being recorded hve. The inputs may be either by a 
user pressing a soft key on a telephone device, or by a voice conference bridge 
determining who are the dominant speakers in a multi-party conference. 

The invention offers an advantage when the indexing metadata is used to 
retrieve the select portions of the telephone conversation. The advantage is that the 
indexing metadata may be used to skip directly to the corresponding select portions of 
the telephone conversation, without having to play out the entire conversation. 

The invention will become more readily apparent from the following Detailed 
Description, which proceeds with reference to the drawings, in which: 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagram for illustrating a voice gateway made according to an 
embodiment of the invention. 

Fig. 2 is a block diagram of a voice gateway of the diagram of Fig. 1 
according to an embodiment of the invention. 

Fig. 3 is a diagram of storage of data taking place in a large memory of the 
diagram of Fig. 1 according to an embodiment of the invention. 

Fig. 4 is a diagram for illustrating a combination of an IP telephone and a 
network switch made according to embodiments of the invention. 

Fig. 5 is a diagram showing special function keys of the telephone of Fig. 4 
according to one embodiment of the invention. 

Fig. 6 is a table showing a memory arrangement of data resulting from 
recording a conversation according to various possible embodiments of the invention. 

Fig. 7 is a diagram of data representing a recorded conversation reflecting the 
table of Fig. 6. 

Fig. 8 is a diagram for illustrating a connection of a conference bridge made 
according to an embodiment of the invention. 

Fig. 9 is a block diagram of a conference bridge of the diagram of Fig. 8 made 
according to an embodiment of the invention. 

Fig. 10 is a table showing a memory arrangement of data resulting from 
recording a voice conference by the conference bridge of Fig. 8. 

Fig. 1 1 is a flowchart illustrating a general method according to an 
embodiment of the present invention. 
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Fig. 12 is a flowchart illustrating a method according to another embodiment 
of the present invention. 

Fig. 13 is a flowchart illustrating a method according to yet another 
embodiment of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S) 

As has been mentioned, the present invention provides devices, softwares and 
methods for generating indexing metadata for select portions of a telephone 
conversation in real time. The conversation is between two or more parties, and at 
least some of it takes place through a packet switched network. The invention is now 
described in more detail. 

Referring now to Fig. 1, an arrangement including embodiments of the 
invention is described. A network 110 operates under an IP protocol, and thus is also 
known as IP Cloud 110. 

A telephone 115 may establish a connection 119 with voice gateway 120 made 
according to an embodiment of the invention. It will be appreciated that a voice 
gateway 120 is used, because telephone 1 15 is an ordinary telephone. Connection 
1 19 is therefore made through the Public Switched Telephone Network (PSTN). 
Equivalently, if an IP telephone were used, then voice gateway 120 might be any 
network switch. 

Telephone 115, with the aid of voice gateway 120, may establish a 
communication link 132 with another voice gateway 140. Voice gateway 140 may 
establish a connection 143 with telephone 146. Accordingly, a user of telephone 115 
may have a conversation with a user of telephone 146 through a combined 
communication link that includes connection 119, communication link 132 and 
connection 143. 

The voice conversation is encoded as digital data that passes through the 
combined communication link. When passing through communication link 132, it is 
in the form of data packets 134, 136, 138. These data packets 134, 136, 138 maybe 
encoded under a Voice over Internet Protocol (VoIP). 

In the embodiment of Fig. 1, a memory 150 is used to store at least portions of 
the conversation, and in some embodiments all of the conversation. This is performed 
by storing in memory 150 groups 164, 166, 168 of digital data, which are also known 
as voice data groups 164, 166, 168, since they may reproduce the conversation. 
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It will be appreciated that memory 150 is preferably implemented by a 
memory that has high capacity^ since large amounts of digital data are typically 
required to represent voice. Accordingly, memory 150 is often called large memory 
or mass memory, to be distinguished from memory registers and incidental memories 
associated with processors. 

Groups 164, 166, 168 may be derived from data that arrives from both 
telephone 115 and telephone 146. When storing voice that was generated from 
telephone 146, groups 164, 166, 168 may be derived from data packets 134, 136, 138 
respectively. Derivation may be in a number of ways, such as by stripping from data 
packets 134, 136, 138 headers that facilitate navigation through IP Cloud 110. That is 
because such headers have fulfilled their mission by the time their encapsulated data 
have reached network switch 120, 

Importantly according to the invention, a data group 172 is generated in real 
time. This means that data group 172 is generated while the conversation is taking 
place. 

Data group 172 does not represent voice data. Data group 172 is metadata, in 
other words data about the voice represented by the associated voice data groups 164, 
166, 168. Data group 172 indexes at least one of the associated voice data groups 
164, 166, 168, for later retrieval. 

Data group 172 is stored in a memory. It may stored in the same memory 150 
as voice data groups 164, 166, 168, although the invention is not limited in this 
regard. 

Referring now to Fig. 2, a network switch 220 made according to an 
embodiment of the invention is described in more detail. Switch 220 may be any 
network switch, such as network switch 420 of Fig. 4, a router, a gateway, voice 
gateway 120 of Fig. 1, a conference bridge such as bridge 810, etc. 

Switch 220 has a network interface 230 for interfacing with a network, such as 
network 110. 

Switch 220 also has a processor 240 coupled with network interface 230. 
Processor 240 may be implemented as a Digital Signal Processor (DSP), Central 
Processing Unit (CPU), or any other equivalent way known in the art. Processor 240 
may include a voice codec 250, which is made from an encoder 253 and a decoder 
256. In other embodiments, codec 250 is not a voice codec. 
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Switch 220 additionally includes a memory 260, on which a program 270 may 
reside. Functions of processor 240 may be controlled by program 270, as will become 
apparent from the below. 

Referring now to Fig, 3, a mapping of stored data is shown inside a memory 
350. Memory 350 may be any memory that stores digital data, such as memory 150 
of Fig. 1, memory 450 of Fig. 4, or memory 850 of Fig. 8. 

In memory 350, groups of voice data 322 are arranged. The arrangement may 
be in a serial fashion, to reflect the order of the represented conversation. In addition, 
incoming data groups 172, 164, 166, 168 are stored. In this embodiment, metadata 
172 is stored along with voice data groups 164, 166, 168. It is also of interest that, in 
the embodiment of Fig. 3, metadata 172 arrives and is stored before voice data groups 
164, 166, 168. The invention is not limited in this regard, and other arrangements are 
also equivalently possible. To save on space, the conversation is recorded as a single 
stream of voice data. The voice data is of the combined voice of the speakers. 

Referring now to Fig. 4, an IP telephone 410 is described, which is made 
according to an embodiment of the invention. IP telephone 410 is for communicating 
through IP cloud 110. 

IP telephone 410 includes a handset that has a microphone 41 1 for receiving 
local voice inputs, and converting them into an analog signal. It also has a speaker 
412 for converting remotely received voice data into sound, after some suitable 
processing of the type that is known in the art. The handset may be provided as part 
of a device 415 that appears as an ordinary telephone. 

Device 415 includes a keyboard with keys for dialing telephone numbers, such 
as an access number for telephone 446. One such key 416 may produce a DTMF 
sound, and may be a number key or a special function key. Usually each key 
produces a different DTMF sound. 

IP telephone 410 additionally includes a signal processing unit 417. Unit 417 
may include an Analog to Digital Converter for converting the output of microphone 
41 1 into local digital voice data. Unit 417 may also include a Digital to Analog 
Converter for converting remote digital voice data into an analog signal suitable for 
speaker 412. 

Unit 417 may also include a network interface for establishing an IP 
connection 419 with a network switch 420 in network 110. Data is exchanged 
through connection 419. In fact, network switch 420 may establish a connection 432 
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with a voice gateway 440 through network 110. And voice gateway 440 may 
establish a connection 443 with an ordinary telephone 446. This way users of 
ordinary telephone 446 and IP telephone 410 may have a conversation. 

Data is received through connection 432 in the form of data packets 434, 436, 
438. These contain remote digital voice data, which encode voice generated at the 
ordinary telephone 446. 

Unit 417 receives the remote digital voice data through switch 420 and link 
419, In addition, unit 417 may store portions of the aggregate local and remote digital 
voice data in a memory 450. In the embodiment of Fig. 4, the stored portions are 
shown as data groups 464, 466, 468, corresponding respectively to data packets 434, 
436, 438. By aggregating the local and remote data, the entire conversation is being 
stored. 

Unit 417 generates indexing metadata about a select portion of the aggregate 
voice data in response to a tagging input. In the embodiment of Fig. 4, the indexing 
metadata is shown as a data group 472. Data group 472 is shovm as being stored in 
an indexing memory 472, distinct from large memory 450 where the conversation 
data is being stored. Once again, that is not necessary for practicing the invention, 
and the indexing data group 472 may be stored in memory 450, depending on the 
configuration. 

In the embodiment of Fig. 4, the tagging input is provided from key 416. That 
may be a tagging signal generated electrically simply from pressing key 416. 
Alternately, the tagging signal may be by a DTMF decoder in unit 417. 

Referring now to Fig. 5, a detail of device 415 is shown according to one 
embodiment of the invention. This could be for a special telephone for a stockbroker. 

In the embodiment of Fig. 5, device 415 has three keys 416-A, 416-B, 416-C. 
These may be used for providing tagging inputs. Key 416-A is used for indexing a 
first portion of the conversation data under a tag of "NAME", key 416-B is used for 
indexing a second portion of the conversation data under a tag of "ACCOUNT 
and key 4 1 6-C is used for indexing a third portion of the conversation data under a tag 
of "TRANSACTION". 

The functions of the keys 416-A, 416-B, 416-C may be conveyed in a number 
of ways. One such way is to have keys 416-A, 416-B, 416-C be hard keys, which 
bear proper inscriptions. Another such way is for them to be soft keys, such as labels 
on icons of keys on a screen. The labels may be the same, or changing dynamically. 
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depending on the flow of the conversation. The soft keys may be selected by a 
pointing device such as a mouse or a "Tab" key, and activated by a mouse button or 
an "Enter" key. In effect, soft keys are programmable by the device. Their behavior 
is configurable. 

Referring to Fig. 6, a table 610 is shown for how tagging may be implemented 
according to an embodiment of the invention. The three keys 416-A, 416-B, 416-C 
are used to generate time instances Tl, T2, T3, T4, T5, T6. These time instances Tl, 
T2, T3, T4, T5, T6 correspond to time, which may be recorded in a way that is 
meaningful to the stream, such as being sequence numbers from the Real Time 
Transport Protocol (RTP). 

In one embodiment, tagging lasts while each of three keys 416-A, 416-B, 416- 
C is pressed. So, when key 416-A is pressed, a time Tl is recorded, under a suitable 
variable name NAME_START. When key 416-A is released, a time T2 is recorded, 
under a suitable variable name NAME_END. Similarly with recording the times T3, 
T4 for key 416-B, and times T5, T6 for key 416-C. 

In another embodiment, each of three keys 416-A, 416-B, 416-C is to be 
pressed and released. The first pressing indicates Tl, T3, T5, respectively, while the 
second pressing indicates T2, T4, T6, respectively. 

Referring now to Fig. 7, a depiction is given of the conversation voice data 
710, which represents a conversation. Data 710 is shown as a stream of data against a 
time axis 715. Voice data 710 is divided into seven segments 721, 722, 723, 724, 
725, 726, 727 by time instances Tl, T2, T3, T4, T5, T6, which are the familiar time 
instances of Fig. 6, generated as described above. Accordingly, segments 722, 724, 
726 correspond respectively to tag durations 742 ("NAME"), 744 ("ACCOUNT #"), 
and 746 ("TRANSACTION"), which were selected as described above with reference 
to Fig. 6. The remaining segments 721, 723, 725, 727 include information of 
comparatively less interest. 

In the embodiment of Fig. 6, Fig. 7, the times Tl, T2, T3, T4, T5, T6, are 
merely time designators along time axis 715. Accordingly, the metadata (group 472) 
that designates times Tl, T2, T3, T4, T5, T6 need not be stored together with 
conversation data 710 (groups 464, 466, 468). In other embodiments, such as the data 
storage arrangement of Fig, 3, the designator group 172 is not necessarily 
denominated by time, but may designate a time by its place within the stream of data. 
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In that case, however, designator 172 is necessarily stored within the conversation 
data groups 322, 164, 166, 168. 

Referring now to Fig, 8, a conference bridge 810 made according to an 
embodiment of the invention is described in more detail. It will be recognized that a 
number of cooperating components are repeated from prior descriptions in a different 
application, for facilitating this illustration. 

Bridge 810 is for placing in a network, such as IP Cloud 110. Bridge 810 is 
adapted to establish communication links through network 110. More specifically, a 
connection 812 is established with voice gateway 140, a cormection 814 is established 
with voice gateway 440, and a connection 816 is established with network switch 840. 
These three links are concurrent in that they are used at the same time, even though 
they may not be established at the same instant, as is known in the art. 

IP phone 847 establishes a connection 843 with network switch 840. This 
way, users of telephones 146, 446 and 843 are capable of having a simultaneous 
conference, through bridge 810. 

It should be noted that, while all three links 812, 814, 816 are through network 
110, that is not necessary for practicing the invention. Indeed, one of them may be 
through an ordinary telephone line. Also, bridge 810 may be located in the same 
place as one of the speakers, in which case the corresponding link need not be through 
a network at all. 

Bridge 810 receives data from all three communication links 812, 814, 816, 
which it considers as channels. In the instant depicted in Fig. 8, bridge 810 is 
receiving voice data packets 862, 864, 866 through link 812, and voice data packets 
842, 844, 846 through link 816. The user of telephone 446 is silent, which therefore 
generates no voice data packets in some embodiments (maybe a silence indication 
packet). 

There are a number of features that conference bridge 810 may have. 

Nominally, bridge 810 sums all the received inputs, and plays them back 
through the channels. This way all users hear an aggregate of what is said, as if they 
were present in the same location. 

As a first improvement, bridge 810 does not play to each user what that user 
said. This way, each user hears an aggregate of only what the others said. 
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As a second improvement, bridge 810 continuously chooses to favor at least 
one of the channels at the expense of disfavoring at least another one of the channels. 
The selection is performed by a channel selector. 

The channel selector analyzes continuously the inputs from the channels, in 
trying to determine which one(s) are the dominant one(s). The determination may be 
by which channel is the loudest. And as the users speak at different times, the channel 
selector changes which chaimel(s) it favors. It may deselect the favored channel, and 
favor another one. This way, at any given time, each user hears only what bridge 810 
deems is the important part of a voice conference. 

In the example of Fig. 8, only link 812 is selected to be favored, and links 814, 
816 are selected to be disfavored. Accordingly, only voice data packets 862, 864, 866 
are transmitted through links 814, 816. Voice data packets 842, 844, 846 are 
discarded. (If, instead, more than one links were selected to be favored, then the sum 
of their voices might be encoded in the transmitted packets.) 

In the embodiment of Fig. 8, the conversation is recorded in a memory 850. 
To save on space, the conversation is recorded as a single stream of voice data. The 
voice data is of the combined voice of those speakers who are favored at the time. 

Memory 850 is large, as discussed above. Memory 850 may be collocated 
with bridge 810 in some embodiments. In other embodiments, memory 850 may be 
accessed by establishing a storage connection 818 with a further network switch 870, 
concurrently with connections 812, 814, 186. Network switch 870 receives voice data 
packets 862, 864, 866, over storage connection 818, and forwards them to memory 
850. These may be further stripped from the headers and trailers for more economical 
storage. 

Importantly according to the invention, bridge 810 generates a selection 
identifier about the selection of which communication link to favor, and which to 
disfavor at any given time. The selection identifier may be encoded as a data group, 
which in turn may be encoded as a packet 872, The selection identifier may include 
an identification code for the party whose channel is favored, and a time stamp of 
when the selection is made. The time stamp may be an RTF time stamp, which may 
then be converted into a different format. Such a different format may 
advantageously be standard user-readable format for retrieval while playback. Such a 
standard may be one of the SMPTE (Society of Motion Picture and TV Engineers) 
standards. 
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Bridge 810 then stores the selection identifier. This may be accomplished by 
transmitting data group 872 to memory 850. Transmitting may be accomplished 
through storage connection 818. Alternately, the selection identifier may be stored in 
a different memory. 

Figure 8 shows one instant of the voice conference. In a subsequent instant, 
the first communication link may be deselected from being favored. Bridge 810 may 
generate and store a deselection identifier about the deselection, which may include a 
time stamp, as per the above. At that, or a later instant, the second communication 
link may be selected to be favored, and the first communication link to be disfavored. 
Again, bridge 8 1 0 may generate and store a selection identifier about the later 
selection. 

Referring now to Fig, 9, a conference bridge 910 made according to an 
embodiment of the invention is described in more detail. Bridge 910 may be any 
conference bridge, such as bridge 810, etc. 

Bridge 910 has a network interface 930 for interfacing with a network, such as 
network 1 10. 

Bridge 910 also has a processor 940 coupled with network interface 930. 
Processor 940 may be implemented as a Digital Signal Processor (DSP), Central 
Processing Unit (CPU), or any other equivalent way known in the art. Processor 940 
may include a codec 950, which is made from an encoder 953 and a decoder 956. 
Codec 950 may or may not be a voice codec. 

Bridge 910 moreover includes a channel selector 958, which may be 
implemented as part of processor 940. Channel selector 958 determines which 
channel(s) will be favored over which channel(s). The input from the disfavored ones 
is ignored. The inputs from the favored ones are added. Channel selector 958 may 
make the determination by employing a speaker selection algorithm, which 
determines who it regards as the dominant speakers. 

Bridge 910 additionally includes a memory 960, on which a program 970 may 
reside. Functions of processor 940 may be controlled by program 970, as will become 
apparent from the below. 

Referring to Fig. 10, a table 1010 is shown for how tagging may be 
implemented by bridge 810 according to an embodiment of the invention. The tinies 
of events, such as selections and deselections may be recorded in a table, using 
suitable variable names, such as START_#1, END_#1, etc. Other variables may also 
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be recorded, such as the ordinal number (first, second, etc.) of when user #1 is 
favored, etc. 

It is readily apparent that the present invention may be implemented by one or 
more devices that include logic circuitry. It may also be implemented by a device that 
includes a dedicated processor system, which may include a microcontroller or a 
microprocessor. 

The invention additionally provides methods, which are described below. 
Moreover, the invention provides apparatus that performs, or assists in performing the 
methods of the invention. This apparatus may be specially constructed for the 
required purposes, or it may comprise a general-purpose computer selectively 
activated or reconfigured by a computer program stored in the computer. The 
methods and algorithms presented herein are not necessarily inherently related to any 
particular computer or other apparatus. In particular, various general-purpose 
machines may be used with programs in accordance with the teachings herein, or it 
may prove more convenient to construct more specialized apparatus to perform the 
required method steps. The required structure for a variety of these machines will 
appear from this description. 

Useful machines or articles for performing the operations of the present 
invention include general-purpose digital computers or other similar devices. In all 
cases, there should be borne in mind the distinction between the method of operating 
a computer and the method of computation itself. The present invention relates also 
to method steps for operating a computer and for processing electrical or other 
physical signals to generate other desired physical signals. 

The invention additionally provides a program, and a method of operation of 
the program. The program is most advantageously implemented as a program for a 
computing machine, such as a general-purpose computer, a special purpose computer, 
a microprocessor, etc. 

The invention also provides a storage medium that has the program of the 
invention stored thereon. The storage medium is a computer-readable medium, such 
as a memory, and is read by the computing machine mentioned above. 

A program is generally defined as a sequence of steps leading to a desired 
result. These steps, also known as instructions, are those requiring physical 
manipulations of physical quantities. Usually, though not necessarily, these quantities 
take the form of electrical or magnetic signals capable of being stored, transferred. 
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combined, compared, and otherwise manipulated or processed. When stored, they 
may be stored in any computer-readable medium. It is convenient at times, 
principally for reasons of common usage, to refer to these signals as bits, data bits, 
samples, values, elements, symbols, characters, images, terms, numbers, or the like. 
It should be borne in mind, however, that all of these and similar terms are associated 
with the appropriate physical quantities, and that these terms are merely convenient 
labels applied to these physical quantities. 

This detailed description is presented largely in terms of flowcharts, display 
images, algorithms, and symbolic representations of operations of data bits within a 
computer readable medium, such as a memory. Such descriptions and representations 
are the type of convenient labels used by those skilled in programming and/or the data 
processing arts to effectively convey the substance of their work to others skilled in 
the art. A person skilled in the art of programming may use this description to readily 
generate specific instructions for implementing a program according to the present 
invention. For the sake of economy, however, flowcharts used to describe methods 
of the invention are not repeated in this document for describing software according 
to the invention. 

Often, for the sake of convenience only, it is preferred to implement and 
describe a program as various interconnected distinct software modules or features, 
collectively also known as software. This is not necessary, however, and there may 
be cases where modules are equivalently aggregated into a single program with 
unclear boundaries. In any event, the software modules or features of the present 
invention may be implemented by themselves, or in combination with others. Even 
though it is said that the program may be stored in a computer-readable mediiam, it 
should be clear to a person skilled in the art that it need not be a single memory, or 
even a single machine. Various portions, modules or features of it may reside in 
separate memories, or even separate machines. The separate machines may be 
connected directly, or through a network, such as a local access network (LAN), or a 
global network, such as the Internet. 

In the present case, methods of the invention are implemented by machine 
operations. In other words, embodiments of the program of the invention are made 
such that they perform methods of the invention that are described in this document. 
These may be optionally performed in conjunction with one or more human operators 
performing some, but not all of them. As per the above, the users need not be 
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collocated with each other, but each only with a machine that houses a portion of the 
program. Alternately, some of these machines may operate automatically, without 
users and/or independently from each other. 

Methods of the invention are now described. 

Referring now to Fig, 1 1 , a flowchart 1 1 00 is used to illustrate a general 
method according to an embodiment of the invention. 

According to a box 1 1 10, a communication link is established through a 
network. The link will be used for receiving voice data in the form of packets. The 
voice data will come in portions, some of which will be tagged. 

According to a next box 1 1 15, it is inquired whether a tagging input has been 
received. 

If not, then according to a next box 1 140, a next portion of voice data is 
received from the communication link. 

If yes, then according to a next box 1 150, a tag is generated about the portion. 
Then according to an optional next box 11 60, the tag is stored, and execution 
continues to box 1 140. 

After box 1 140, according to an optional next box 1 170, the received portion 
of voice data is converted into sound. 

According to an optional next box 1 1 80, it is again whether a tag was 
generated. This is substantially the same question as was asked in step 1115, If not, 
then execution may return to box 1 1 1 5, to ask whether a next tagging input was 
received. 

If yes, then according to a next box 1 190, the received portion of voice data is 
stored in a large memory. Execution then returns to box 1115. 

Accordingly, if box 1 180 has taken place, then only the tagged portions are 
thus stored. If box 1 180 does not take place, then all the portions are stored, whether 
tagged or not. 

Referring now to Fig. 12, a flowchart 1200 is used to illustrate a method 
according to another embodiment of the invention. As per the above, the method of 
flowchart 1200 may be performed by an IP phone, etc. 

According to a box 1210, a communication link is established through a 
network. The linlc will be used for receiving voice data in the form of packets. The 
voice data will come in portions, some of which will be tagged. 
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According to a next box 1215, it is inquired whether a tagging input has been 
received. As per the above, the tagging input may be a DTMF signal, etc. If multiple 
indexing features are enabled (e.g. one for "NAME", one for "ACCOUNT etc.), 
then it is not yet known which one was activated. 

If yes, then according to a next box 1220, it is inquired whether a first 
indexing feature was activated by the tagging input. If yes, then according to a next 
box 1223, a first tag is generated about the voice data. According to a next box 1226, 
the first tag is stored. 

If, at box 1220, it is determined that the first feature was not activated, then, 
according to a next box 1230, it is inquired whether a second indexing feature was 
activated by the tagging input. If yes, then according to a next box 1233, a second tag 
is generated about the voice data. According to a next box 1236, the second tag is 
stored. 

According to a next box 1240, a portion of the next voice data is received. 
This takes place if no tagging input is received at box 1215, or after box 1226, or after 
box 1236. 

According to an optional next box 1250, the received portion of voice data is 
converted into sound. 

According to a next box 1260, the received portion of voice data is stored in a 
large memory. Execution then returns to box 1215. 

Referring now to Fig. 13, a flowchart 1300 is used to illustrate a method 
according to yet another embodiment of the invention. As per the above, the method 
of flowchart 1300 may be performed by a conference bridge, etc. 

According to a box 1310, at least three communication links are established. 
Of those, at least the first and the second are through a network. 

According to an optional next box 1320, three streams of voice data are 
received from the three communication links, one from each. Sometimes, the voice 
data may contain zero voice, or be a Silence Indication Packet. 

According to an optional next box 1330, the first communication link is 
selected for favoring, while the second communication link is selected for 
disfavoring. The determination may be from the content of the streams of voice data. 

According to an optional next box 1340, it is inquired whether this was the 
first selection. 
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If yes, then according to a next box 1350, a selection identifier is generated 
and stored. The selection identifier may identify the party of the first communication 
link. It may also include an RTP time stamp. The RTP time stamp may then be 
converted to a different format, such as an SMPTE format. 

If this were not the first selection then according to a next box 1360, it is 
inquired whether the selection is different from the immediately previous one. If yes, 
then the new selection includes a deselection of the previous selection. According to 
an optional next box 1370, a deselection identifier is generated and stored. It may 
include an RTP time stamp, which may then be converted to a different format. 
Execution then proceeds to box 1350. 

If, at box 1360, the selection is not different from the previous one, or after 
box 1350, then, according to an optional next box 1380, the first but not the second 
voice data is retransmitted through the third communication link, and also optionally 
through the second communication link. The transmission of box 1380 may also 
include voice data from yet another speaker who is favored over the second voice 
data. 

According to an optional next box 1390, the first but not the second voice data 
is stored in a large memory. Execution then returns to box 1320. 

The devices, softwares and methods of the present invention are used for real 
time recording. The indexing tags of the invention make it easy to scan a recorded 
conversation or conference for items of interest. The indexing tags are used by a 
playback device to skip to either an indexed portion of a conversation, or to instances 
of speech of a desired speaker. In a generic application, the playback device would 
include a "PREVIOUS INDEX" and a "NEXT INDEX" buttons. 

A person skilled in the art will be able to practice the present invention in view 
of the description present in this document, which is to be taken as a whole. 
Numerous details have been set forth in order to provide a more thorough 
understanding of the invention. In other instances, well-known features have not been 
described in detail in order not to obscure unnecessarily the invention. 

While the invention has been disclosed in its preferred form, the specific 
embodiments as disclosed and illustrated herein are not to be considered in a limiting 
sense. Indeed, it should be readily apparent to those skilled in the art in view of the 
present description that the invention may be modified in numerous ways. The 
inventor regards the subject matter of the invention to include all combinations and 
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subcombinations of the various elements, features, functions and/or properties 
disclosed herein. 

The following claims define certain combinations and subcombinations, which 
are regarded as novel and non-obvious. Additional claims for other combinations and 
subcombinations of features, functions, elements and/or properties may be presented 
in this or a related document. 
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